While you might know Kagi best as the paid competitor to Google's ever-worsening search product, the company launched its Kagi Translate tool back in 2024, saying at the time that it was a 'simply better' competitor to tools like Google Translate and DeepL. At launch, the company said Kagi Translate 'uses a combination of LLMs, selecting and optimizing the best output for each task,' a fact that 'can occasionally lead to quirks that we're actively working to resolve.'
Our measure, 'observed exposure,' compares the tasks LLMs are theoretically capable of to the tasks people actually use Claude for at work. We find that actual usage is far from reaching theoretical capability.